System and Methods for Distributed Machine Learning with Multiple Data Sources, Multiple Programming Languages or Frameworks, and Multiple Devices or Infrastructures

ABSTRACT

Methods and systems are presented for consuming different data sources, and deploying artificial intelligence and machine learning programs on different target devices or infrastructures. Many data types can be transformed into machine learning data shards (MLDS) while many machine learning programs written in various programming languages or frameworks are transformed to common operator representations. Operator representations are transformed into execution graphs (EG) for a chosen target device or infrastructure. The MLDS and EG are input to the targeted devices and infrastructures, which then execute the machine learning programs (now transformed to EGs) on the MLDS to produce trained models or predictions with trained models.

FIELD OF THE INVENTION

The present invention generally relates to a distributed computingsystem for artificial intelligence (AI) and machine learning (ML)programs written in multiple programming languages or frameworks, andmore particularly, is directed to a method of transforming AI and MLprograms into common operator representations (OR) for generatingexecution graphs (EG) for target devices or infrastructures of differentcomputing paradigms (e.g., single laptop, distributed server or Internetof things (IoT) clusters), and further, a method of consuming datasources, such as local or cloud data with different formats, databasetables, health records, transaction logs, images and videos, with astandardized interface to facilitate the aforementioned method.

BACKGROUND

Artificial intelligence (AI) and machine learning (ML) applications havebecome increasingly popular in clouds and enterprise data-centers. AIand ML programs are deployed to address problems in different domains,such as medical diagnosis, vehicle self-driving, risk management, imageand video perception, natural language understanding, etc. Each domainmay produce data in different formats, such as website feeds, databasetables, health records and financial transaction logs, various device(e.g., manufacturing, vehicle, and Internet of things (IoT)) logs withaudio, images and videos. The data from these data sources can provideraw data for AI/ML, programs.

AI programs are often written in different programming languages, suchas Python, Lua and C++, and with different ML frameworks, such asTensorflow, Caffe and Torch. Each program or framework usuallyimplements common ML algorithms and models. However, each implementationof these algorithms and models has its own characteristics, and oftendefects, which must be maintained and debugged independently. Moreover,each program or framework may have its own formatting requirements forinput data.

AI programs often target a variety of devices or infrastructures ofdifferent computing paradigms. For example, AI programs may targetdevices and infrastructures such as individual workstations and laptops,distributed servers, and IoT clusters, which may be either on-premisesor in the cloud. These target devices and infrastructures may havedifferent underlying OS and hardware architectures, such as Linux,Windows, x86 or ARM CPU, NVIDIA or AMD GPU, FPGA, ASIC, Ethernet orInfiniBand etc., and may be used in different scenarios, such as forproducing models by training and for inferring predictions using trainedmodels.

Each of the different data sources, different programming languages orframeworks, and different target devices or infrastructures describedabove contribute to the complexity of deploying AI and ML solutions.Conventional native implementations capable of addressing K number ofdata sources, L number of programming languages or frameworks, and Mnumber of target devices or infrastructures would result in up to K×L×Mimplementation combinations, which dramatically increases the cost ofthe overall system, and is prone to produce inconsistent andnon-repeatable results.

SUMMARY OF THE INVENTION

The presently disclosed embodiments are directed to solving issuesrelating to one or more of the problems presented in the prior art, aswell as providing additional features that will become readily apparentby reference to the following detailed description when taken inconjunction with the accompanying drawings.

One embodiment is directed to a distributed computing system forartificial intelligence (AI) and machine learning (ML) systems,comprising an Omni-Source System (OmSS), an Omni-Lingual System (OmLS),and an Omni-Mount System (OmMS). A Data Identification/Sharding Module(DISM) in the OmSS can receive data, generate a data signature, anddivide the data into a number of data pieces. One or more Data EngineModules (DEM) in the OmSS can transform the data pieces into machinelearning data shards by modifying the data pieces based on the datasignature. A Database System (DbS) in the OmSS can combine the machinelearning data shards into a stored machine learning data shards record.The OmLS can include a parser module (PM) which can receive programcode, parse the program code into a program code parse tree, and createan operator representation of the program code. The OmMS can include anExecution Graph Generator Module (EGGM), which can create an executiongraph, and create a hardware-specialized execution graph by transformingthe execution graph based on target device information received from auser. The hardware-specialized execution graph be sent to the one ormore target devices.

Another embodiment is directed to a method of representing data from aplurality of data sources in a consistent format. Data is received froma data source. A data signature can be determined based on data source.The data can be divided into a plurality of data pieces and distributedto a respective data engine machines. Each data machine can transform arespective data piece based on the data signature into a machinelearning data shard. Finally, the machine learning data shards from eachdata engine machine are combined into a machine learning data shardsrecord.

Another embodiment is directed to a method of running a plurality ofmachine learning programs written in different programming languages onmultiple target devices. Target device information and a plurality ofmachine learning programs are received. A program parse tree can begenerated based on the program, and an operator representation of theprogram can be generated by substituting functions from a mapping tablethat are found in the program parse tree with corresponding mathematicaloperators from the mapping table. The operator representation of theprogram can be converted into an execution graph of the program bygenerating one or more graph nodes and one or more relationships betweenthe graph nodes. Hardware specifications are loaded based on the targetdevice information, and the execution graph can be transformed into ahardware-specialized execution graph based on the hardwarespecifications. Finally, the hardware-specialized execution graph is runon the target device.

Further features and advantages of the present disclosure, as well asthe structure and operation of various embodiments of the presentdisclosure, are described in detail below with reference to theaccompanying drawings.

BRIEF DESCRIPTION OF THE DRAWINGS

The present disclosure, in accordance with one or more variousembodiments, is described in detail with reference to the followingfigures. The drawings are provided for purposes of illustration only andmerely depict exemplary embodiments of the disclosure. These drawingsare provided to facilitate the reader's understanding of the disclosureand should not be considered limiting of the breadth, scope, orapplicability of the disclosure. It should be noted that for clarity andease of illustration these drawings are not necessarily made to scale.

FIG. 1 is a block diagram showing an exemplary distributed computingsystem including an Omni-Source System (OmSS), an Omni-Lingual System(OmLS), and an Omni-Mount System (OmMS), which can be implementedaccording to embodiments of the invention;

FIG. 2 presents an exemplary process of converting example data from adata source into machine learning data shards, and further into amachine learning data shards record according to embodiments of theinvention;

FIG. 3 presents an exemplary process of converting example ML programcode into an ML operator representation, and further, into an executiongraph according to embodiments of the invention;

FIG. 4 is an exemplary algorithm of processing a programming requestaccording to embodiments of the invention; and

FIG. 5 is a block diagram illustrating configurations of a computer inwhich embodiments of the invention can be implemented.

DETAILED DESCRIPTION OF EXEMPLARY EMBODIMENTS

The following description is presented to enable a person of ordinaryskill in the art to make and use the invention. Descriptions of specificdevices, techniques, and applications are provided only as examples.Various modifications to the examples described herein will be readilyapparent to those of ordinary skill in the art, and the generalprinciples defined herein may be applied to other examples andapplications without departing from the spirit and scope of theinvention. Thus, embodiments of the present invention are not intendedto be limited to the examples described herein and shown, but is to beaccorded the scope consistent with the claims.

The word “exemplary” is used herein to mean “serving as an example orillustration.” Any aspect or design described herein as “exemplary” isnot necessarily to be construed as preferred or advantageous over otheraspects or designs.

Reference will now be made in detail to aspects of the subjecttechnology, examples of which are illustrated in the accompanyingdrawings, wherein like reference numerals refer to like elementsthroughout.

It should be understood that the specific order or hierarchy of steps inthe processes disclosed herein is an example of exemplary approaches.Based upon design preferences, it is understood that the specific orderor hierarchy of steps in the processes may be rearranged while remainingwithin the scope of the present disclosure. The accompanying methodclaims present elements of the various steps in a sample order, and arenot meant to be limited to the specific order or hierarchy presented.

Embodiments disclosed herein are related to a distributed computingsystem for enterprise artificial intelligence (AI) programs, where thesystem is configured to enable a variety of AI programs to consume avariety of data sources, and to generate efficient executables capableof running on a variety of target devices and infrastructures. Theinventive system can address K number of data sources, L number ofprogramming languages or frameworks, and M number of target devices orinfrastructures, with up to K×L×M implementation combinations, whichsignificantly lowers the cost of the overall system that producesstandardized and repeatable results.

FIG. 1 is a block diagram showing an exemplary distributed computingsystem 100 including an Omni-Source System (OmSS) 130, an Omni-LingualSystem (OmLS) 140, and an Omni-Mount System (OmMS) 150, which can beimplemented according to embodiments of the invention. Each of the OmSS,OmLS, and OmMS may operate on a separate machine or virtual machine. Asa high level summary, the distributed computing system 100 can operateas follows: (1) the OmSS 130 converts data from a variety of datasources 110 into machine data learning shards (MLDS) 137, which are thenstored in an ML Database System (DbS) 138; (2) the OmLS 140 converts MLprograms 130 having a variety of programming languages into operatorrepresentations 145 (also referred to as ML model operatorrepresentations) of the program code; and (3) the OmMS receives thestored MLDS 137 and operator representations of the ML program 130,converts the ML operator representations 145 into a hardware-specializedexecution graphs (HSEG) 157, and distributes the MLDS and HSEG to avariety of target systems and infrastructures 160. A detaileddescription of the computing system 100 and its operation is givenbelow.

As illustrated in FIG. 1, the Omni-Source System (OmSS) 130 isconfigured to receive data from a variety of data sources 110, which mayinclude (but are not limited to) websites or web feeds, databases ortables, records such as electronic health records, and video, audio, andsensor feedback. The OmSS 130 may operate in batch mode, where the datasource already contains all needed data, and streaming mode, where thedata from the data source 110 is generated and received by the OmSS inreal time.

The OmSS 130 first reads the data from the data source 110 through aData Identification/Sharding Module (DISM) 132, which computes a datasignature based on the data source. This data signature identifies thetype of data (website, table, etc.) contained in the data source. Insome examples, the data signature may be computed based on thecharacteristics of the data, while in other examples, the data signaturemay be computed based on an input provided by a user. Once the datasignature is computed, the data from the data source 110 is then evenlydivided into P pieces, and distributed to P machines (e.g., machine 134)along with the data signature. Each of the P machines may include a DataEngine Module (DEM) 136 that converts a respective piece of the data toan ML Data Shard (MLDS) 137 by applying a filter, the filter beingchosen based on the data signature. The MLDS 137 (e.g., theD-dimensional vectors) are then sent to the ML Data Database System(DbS) 138, which combines or concatenates the respective MLDS 137 fromeach of the DEM machines 134 and stores them for later use. This storedcombination of individual MLDS 137 is referred to throughout the presentdisclosure as an MLDS record. In some examples, an MLDS record is anN-by-D matrix where each of the N rows is a D-dimensional vector thatrepresents a single datum, such as the text from a webpage, a singleframe of video, or a single patient's electronic health record.

FIG. 2 illustrates an example process of converting data from a datasource 210 into machine learning data shards 220, and further into amachine learning data shards record 240. In the example shown in FIG. 2,the data from data sources 210 includes individual webpages from W(1) toW(N). In this example, if the data signature corresponds to “webpages”,the DEM (e.g., DEM 136) can, for example, apply a TF-IDF (TermFrequency, Inverse Document Frequency) filter to transform each webpageW(1)-W(N) into MLDS 220, where the MILDS is in the form of aD-dimensional vector. As shown, the MLDS 220 (e.g., D-dimensionalvectors) are then sent to the ML Database System (DbS) 138, whichcombines the respective MLDS 137 from each of the DEM machines 136 intoan MLDS record 240 having the form of an N-by-D matrix, which is storedfor later use.

Referring back to FIG. 1, in order to handle a variety of datasignatures, the DEM 136 may be pre-programmed with a library of filtersthat are known in the art (such as TF-IDF, Principal Component Analysis,etc.), which may be selected based on the data signature. In addition,in some configurations, the library of filters in the DEM 136 may becapable of being updated by a user in order to add additional filters tothe system.

As also shown in FIG. 1, the Omni-Lingual System (OmLS) 140 can convertML program 130 written in different programming languages usingdifferent frameworks into an ML operator representation 145. Examples ofprogramming languages capable of being received by the OmLS 140 includePython, C++, and Java, and example ML frameworks include TensorFlow andCaffe. In one embodiment, a Parser Module (PM) 144 in the OmLS 140 isresponsible for converting ML programs 130 written in variousprogramming languages, into an ML operator representation 145. In theseconfigurations, the PM 144 can contain a mapping table for eachsupported programming language. The mapping table can have two columns:the first column containing functions or patterns from the supportedlanguage, and the second column containing that function or pattern'scorresponding mathematical operator. When the PM 144 is input with an MLprogram 130, it loads the mapping table corresponding to the programminglanguage that the ML program is written in. The PM 144 can use standardcompiler parsing techniques to generate a parse tree from the ML program130. The PM 144 can then sweep over every element of the parse tree,substituting all functions or patterns that match the first column ofthe mapping table with their corresponding operator in the secondcolumn. In some configurations, all elements of the parse tree that didnot match the first column are discarded. The end result is a parse treeof operators, which is referred to herein as the ML operatorrepresentation 145 of the ML program 130.

The Omni-Mount System (OmMS) 130 shown in FIG. 1 includes a machine 142which converts the ML operator representation 145 of the ML program 130into a hardware-specialized execution graph (HSEG) 157, which can be runon a variety of target systems 160. Target systems 160 may include avariety of devices or infrastructures of different computing paradigms,including (but not limited) to workstations (e.g., workstation 162shown), a datacenter machine (e.g., datacenter machine 164), or anInternet of Things (IoT) device, (e.g., IoT device 166 shown). In someembodiments, machine 154 includes an Execution Graph Generator Module(EGGM) 155, which receives the ML operator representation 145 from theParsing Module (PM) 144 along with target hardware information 158. Insome cases, the target hardware information 158 may be computed based ondetected characteristics of target systems 160, while in other cases,the target hardware information may be provided by a user. The EGGM 155is configured to create an execution graph by generating one or moregraph nodes and graph node relationships based on the ML operatorrepresentation 145. In some embodiments, the execution graph may betransformed or refined to create a hardware-specialized execution graph(HSEG) 157 based on the target hardware information 158. In someembodiments, transforming the execution graph into a HSEG can includepartitioning the execution graph such that the HSEG is configured to runin parallel on one or more target systems 160. The conversion from theML operator representation 145 into the HSEG 157 is explained in moredetail below with reference to the example shown in FIG. 3.

Once created, the HSEG 157 is sent by the EGGM 155 in the OmMS 150 toone or more target systems 160. In some embodiments, the EGGM 155 may befurther configured to create hardware-specialized executables (alsoreferred to as graph execution modules) based on the target hardwareinformation, which enable a hardware-specialized execution graph 157 torun on a specific target system. For example, the EGGM 155 may beconfigured to generate and send a workstation graph execution module 163to a workstation 162, a datacenter graph execution module 165 to adatacenter machine 164, an IoT graph execution module 167 to an IoTdevice 166, and so on.

In some embodiments, the OmMS 150 can also include a machine 152 havinga Data Partitioning Module (DPM) 153 configured to retrieve the MLDS 137from the OmMS (e.g., a MLDS record from the ML Database System (DbS)138), partition the MLDS into a plurality of MLDS pieces, and distributethe MLDS pieces to the one or more target systems 160. In this way, theOmMS 150 can match specific MILDS pieces to suitable target systems 160,for example, based on the data signature.

FIG. 3 illustrates an exemplary process 300 of converting example MLprogram code 310 into an example ML operator representation 320, andfurther, into an example hardware-specific execution graph 330 accordingto embodiments of the invention. As shown, the example program code 310creates three “Equation” objects “a,” “b,” and “f,” which incorporatethree “Symbol” objects “W,” “x,” and “c.” Example program code 310 maycorrespond, for example, to program code of ML programs 130 discussedwith reference to FIG. 1 above. As shown in FIG. 3, the example programcode 310 is converted into an ML operator representation 320 using, forexample, the Parsing Module (PM) 144 in the OmLS 140 discussed above.Next, the ML operator representation 145 is converted to an executiongraph, for example, using the Execution Graph Generator Module (EGGM)155 in the OrnMS 150 shown in FIG. 1. Although not shown in FIG. 3, theexecution graph may have a form similar to that of thehardware-specialized execution graph 330 shown, where symbols andfunctions are mapped to graph nodes (e.g., symbol W to node 242 andsymbol x to node 244) and their relationships are mapped to graph noderelationships (e.g., relationship 246).

As discussed with reference to FIG. 1 above, depending on the hardwarespecifications of a target device or computing paradigm (e.g., x86 orARM CPU, GPU, Ethernet, InfiniBand), the execution graph may betransformed in order to create a hardware-specialized execution graph330, for example, using the EGGM 155 in FIG. 1. In some cases,transforming the execution graph into a hardware-specialized executiongraph includes limiting certain symbols or functions according to one ormore device limitations specified in the target hardware information(e.g., target hardware information 158 in FIG. 1). For example, given anembedded system, the OmMS may specify that the mathematical symbols Wand x be restricted to 16-bit floating point storage or quantized to8-bit integer storage, in order to match the embedded CPU'scapabilities. In another example, given a datacenter, the OmMS mayspecify that the EG is to be partitioned across different machines usinga standard graph partitioning algorithm such as METIS, in order toexploit parallelism opportunities. In some embodiments, the executiongraph (not shown) is not modified or transformed, and thus, thehardware-specialized execution graph 330 is essentially the same as theexecution graph.

FIG. 4 is an exemplary algorithm 400 of processing a programming requestaccording to embodiments of the invention. The components discussed withreference to FIG. 4 may correspond, for example, to those samecomponents shown in FIG. 1. In step 432, a user inputs data from a datasource into the Omni-Source System (OmSS). At step 434, a DataIdentification/Sharding Module (DISM) computes a data signature of thedata source, and splits the data over multiple machines. At step 436, aData Engine Module (DEM) in each machine applies a filter to the splitdata according to its data source signature. At step 438, the DataEngine Modules output ML Data Shards (MLDS) to be stored in an MLDatabase System (DbS). As indicated in FIG. 4, steps 432-438 can beperformed by the OmSS machines 430.

At step 442, a user inputs ML program code to the Omni-Lingual System(OmLS). At step 444, the OmLS ML Parsing Module (PM) converts ML programcode into an ML operator representation. As indicated in FIG. 4, steps442-444 can be performed by the OmLS machines 440.

At step 452, a user inputs target hardware information to the Omni-MountSystem (OmMS). At step 454, an Execution Graph Generator Module (EGGM)reads the ML operator representation from the OmLS. At step 456, theEGGM converts the operator representation into an execution graph. Atstep 458, EGGM optimizes or partitions the execution graph according totarget hardware information. At step 459, EGGM outputs ahardware-specialized execution graph to target computing machinesmatching the target hardware information. As indicated in FIG. 4, steps452-459 can be performed by the OmMS machines 450.

At step 462, target computing machines receive target hardwareinformation from EGGM in the OmSS. At step 464, target computingmachines read MLDS from ML DbS in the OmSS. At step 466, targetcomputing machines may now run the ML program, now converted to ahardware-specialized execution graph, on the data source, now convertedto ML data shards. As indicated in FIG. 4, steps 462-466 can beperformed by target machines 460.

FIG. 5 is a block diagram illustrating configurations of a computer 10in which embodiments of the invention can be implemented. Computer 10can perform any of the methods described with reference to FIGS. 1-4above. Computer 10 can include one or more processors (CPU) 11, storage(memory) 12, an input unit 13, display unit 14, and network interface(I/F) 15 configured to interface with a network 20. These components mayinterface one with another via a bus 16. Applications 17 may be storedon memory 12 and may include data and instructions for performing any ofthe methods described in this disclosure, including those described withreference to FIGS. 1-4.

While various embodiments of the invention have been described above, itshould be understood that they have been presented by way of exampleonly, and not by way of limitation. Likewise, the various diagrams maydepict an example architectural or other configuration for thedisclosure, which is done to aid in understanding the features andfunctionality that can be included in the disclosure. The disclosure isnot restricted to the illustrated example architectures orconfigurations, but can be implemented using a variety of alternativearchitectures and configurations. Additionally, although the disclosureis described above in terms of various exemplary embodiments andimplementations, it should be understood that the various features andfunctionality described in one or more of the individual embodiments arenot limited in their applicability to the particular embodiment withwhich they are described. They instead can be applied alone or in somecombination, to one or more of the other embodiments of the disclosure,whether or not such embodiments are described, and whether or not suchfeatures are presented as being a part of a described embodiment. Thusthe breadth and scope of the present disclosure should not be limited byany of the above-described exemplary embodiments

In this document, the term “module” as used herein, refers to software,firmware, hardware, and any combination of these elements for performingthe associated functions described herein. Additionally, for purpose ofdiscussion, the various modules are described as discrete modules;however, as would be apparent to one of ordinary skill in the art, twoor more modules may be combined to form a single module that performsthe associated functions according embodiments of the invention.

In this document, the terms “computer program product”,“computer-readable medium”, and the like, may be used generally to referto media such as, memory storage devices, or storage unit. These, andother forms of computer-readable media, may be involved in storing oneor more instructions for use by processor to cause the processor toperform specified operations. Such instructions, generally referred toas “computer program code” (which may be grouped in the form of computerprograms or other groupings), when executed, enable the computingsystem.

It will be appreciated that, for clarity purposes, the above descriptionhas described embodiments of the invention with reference to differentfunctional units and processors. However, it will be apparent that anysuitable distribution of functionality between different functionalunits, processors or domains may be used without detracting from theinvention. For example, functionality illustrated to be performed byseparate processors or controllers may be performed by the sameprocessor or controller. Hence, references to specific functional unitsare only to be seen as references to suitable means for providing thedescribed functionality, rather than indicative of a strict logical orphysical structure or organization.

Terms and phrases used in this document, and variations thereof, unlessotherwise expressly stated, should be construed as open ended as opposedto limiting. As examples of the foregoing: the term “including” shouldbe read as meaning “including, without limitation” or the like; the term“example” is used to provide exemplary instances of the item indiscussion, not an exhaustive or limiting list thereof; and adjectivessuch as “conventional,” “traditional,” “nominal,” “standard,” “known”,and terms of similar meaning, should not be construed as limiting theitem described to a given time period, or to an item available as of agiven time. But instead these terms should be read to encompassconventional, traditional, normal, or standard technologies that may beavailable, known now, or at any time in the future. Likewise, a group ofitems linked with the conjunction “and” should not be read as requiringthat each and every one of those items be present in the grouping, butrather should be read as “and/or” unless expressly stated otherwise.Similarly, a group of items linked with the conjunction “or” should notbe read as requiring mutual exclusivity among that group, but rathershould also be read as “and/or” unless expressly stated otherwise.Furthermore, although items, elements or components of the disclosuremay be described or claimed in the singular, the plural is contemplatedto be within the scope thereof unless limitation to the singular isexplicitly stated. The presence of broadening words and phrases such as“one or more,” “at least,” “but not limited to”, or other like phrasesin some instances shall not be read to mean that the narrower case isintended or required in instances where such broadening phrases may beabsent.

Additionally, memory or other storage, as well as communicationcomponents, may be employed in embodiments of the invention. It will beappreciated that, for clarity purposes, the above description hasdescribed embodiments of the invention with reference to differentfunctional units and processors. However, it will be apparent that anysuitable distribution of functionality between different functionalunits, processing logic elements or domains may be used withoutdetracting from the invention. For example, functionality illustrated tobe performed by separate processing logic elements or controllers may beperformed by the same processing logic element or controller. Hence,references to specific functional units are only to be seen asreferences to suitable means for providing the described functionality,rather than indicative of a strict logical or physical structure ororganization.

Furthermore, although individually listed, a plurality of means,elements or method steps may be implemented by, for example, a singleunit or processing logic element. Additionally, although individualfeatures may be included in different claims, these may possibly beadvantageously combined. The inclusion in different claims does notimply that a combination of features is not feasible and/oradvantageous. Also, the inclusion of a feature in one category of claimsdoes not imply a limitation to this category, but rather the feature maybe equally applicable to other claim categories, as appropriate.

1-9. (canceled)
 10. A method of representing data from a plurality ofdata sources in a consistent format, comprising: reading data from adata source of the plurality of data sources; determining a datasignature based on the data source; dividing the data into a pluralityof data pieces; distributing each data piece to a respective data enginemachine of a plurality of data engine machines; selecting, at each dataengine machine, one or more filters based on the data signature;transforming, at each data engine machine, a respective data piece ofthe plurality of data pieces into a machine learning data shard usingthe one or more filters; and combining the respective machine learningdata shards from each data engine machine into a machine learning datashard database record.
 11. The method of claim 10, wherein each machinelearning data shard from a respective data engine machine is representedas a vector having a number of dimensions, D.
 12. The method of claim11, wherein the machine learning data shard database record isrepresented as a matrix having N rows and D columns.
 13. The method ofclaim 10, wherein the data source is a streaming data source andreceiving the data includes continually receiving streaming data fromthe streaming data source.
 14. The method of claim 10 furthercomprising: reading a second set of data from a second data source ofthe plurality of data sources; determining a second data signature basedon the second data source; transforming a respective second data pieceof a plurality of second data pieces into a second machine learning datashard using one or more second filters, and combining the respectivesecond machine learning data shards into a second machine learning datashard database record, wherein the second machine learning data sharddatabase record has the same consistent format as the machine learningdata shard database record.
 15. The method of claim 10, furthercomprising: partitioning the machine learning data shard database recordinto a plurality of machine learning data shard record pieces, anddistributing the machine learning data shard record pieces to one ormore target devices. 16-20. (canceled)